The invention concerns a method and an apparatus for determining the phase and amplitude information of an electromagnetic wave.
The term phase here generally stands for phase transit time and for the designation transit time which is also used according to the respective signal shape involved.
Hereinafter reference is made to a light wave instead of an electromagnetic wave. That however does not denote a restriction only to the spectral range of visible electromagnetic waves, but is only for the purposes of simplification.
For the measurement of frequency components in terms of amplitude and phase in wide-band and high-frequency signals, the electronic measuring art and communication art frequently use phase detectors which multiply or mix the unknown signal with a sine oscillation and determine the steady component which occurs in the presence of a signal component of the same frequency by integration or low-pass filtering.
That procedure produces the correlation function of the unknown signal with the mixing signal for a given, adjustable relative phase position. By altering the mixing frequency (sweep) the unknown signal can be broken down into its spectral components. Steady component, varying amplitude and phase of the unknown frequency component of the same frequency can be determined by at least three phase positions.
The investigation of corresponding optical signals which have acquired increasing significance in the measuring and communication arts is implemented nowadays inter alia by way of wide-band photodetectors as electro-optical transducers with subsequent electronic measurement value ascertainment, as previously described for electrical signals.
Because of the high level of expenditure involved those methods and the corresponding measurement apparatuses are usually only of a one- or two-channel nature. In the case of optical signals however very many parallel channelsxe2x80x94in particular entire image sequencesxe2x80x94with high frequency components frequently have to be surveyed simultaneously.
Besides the spectral modulation properties of two-dimensional light waves, an aspect of increasing interest is the rapid run of the envelope in space and time. In addition there is a wish to provide for rapidly and accurately surveying 3D-objects, for example by way of optical radar processes, which requires very fast detectors in the sub-nanosecond range, as a result of the light speed of the echo signals. At the same time they should be available as a detector array if there is desire to avoid the time-consuming operation of scanning the actively or passively bright 3D-objects.
Such a 3D-camera is proposed in DE 44 39 298 A1 which the present invention takes as its basic starting point.
FIG. 10 is intended to illustrate that 3D-camera which is based on the echo transit time or phase transit time process. The HF-modulated light wave 101 which is irradiated by a modulated light transmitter 107 and 103 and reflected by the 3D-object 100 contains all the depth information in the delay in respect of the phase front. If the incident light wave is modulated once again in the reception aperture 102 with a two-dimensional, optical mixer 104 of the same frequency, which corresponds to a homodyne mixing or demodulation process, the result is a steady high-frequency interferogram.
That HF-interferogram can be recorded with a conventional CCD-camera 105 and subjected to further processing with an image processing arrangement 106. Integration of the steady component of the mixed product in the CCD-photoelectric charge corresponds to the formation of the correlation function of the two mixing signals. The distance-related phase delays due to the echo transit times and the amplitudes can be calculated pixel-wise from three or more interferograms by virtue of different phases of the demodulating mixing frequency, for example 0xc2x0, 120xc2x0 and 240xc2x0 or 0xc2x0, 90xc2x0, 180xc2x0 and 270xc2x0, and thus the 3D-depth image can be reconstructed.
The two-dimensional optical mixer 103 or 104 which is also referred to as a spatial light modulator or SLM comprises in that case for example a Pockel cell which has a series of serious disadvantages which are described in the literature.
Further implementation options are afforded by LCD-windows which are admittedly inexpensive but which are about a factor of 1000 too low in terms of the desired band width.
The use of a so-called microchannel plate, as is used in image amplifiers, is also expensive and costly. The gain can be modulated by modulation of the acceleration voltage which is applied to the microchannels and which influences the secondary electron emission in the microchannels.
Furthermore, the state of the art sets out a proposal for a 2D-correlator based on a CCD-photodetector array: xe2x80x9cThe Lock-In CCD-Two-Dimensional Synchronous Detection of Lightxe2x80x9d by Spirig, Seitz et. al., published in IEEE Journal of Quantum Electronics, Vol. 31, No. 9, September 1995, pages 1705-1708. There, a photopixel is interrogated by way of four transfer gates in order to ascertain the phase of sine-modulated light. For each sine period, a respective equidistant sample is taken with the four transfer gates, whereby the phase can be easily calculated. That procedure is too slow for the indicated problems as the harmonic light signal is firstly integrated on during a scanning duration which significantly delimits the band width. It is only then that the desired mixing process is implemented with the stored charge being taken over as the scanning sample.
The object of the present invention is therefore that of providing a method and an apparatus for determining the phase and/or amplitude information and thus the envelope of a light wave, which permit a simpler, wider-band and less expensive correlator concept and rapid 3D-object surveying by way of a predeterminable lighting.
The above-indicated object is now attained by the method as set forth in claim 1 and by the photonic mixing element as set forth in claim 14, by the mixing element arrangement set forth in claim 20 and by the apparatus set forth in claim 23.
The principle according to the invention s based on a drift produced by the modulation photogate voltage and separation of the minority charge carriers photo-generated by the light wave in the material beneath at least two adjacent light-sensitive modulation photogates. In this case those charge carriers drift under the influence of the modulation photogate voltages Uam(t) and Ubm(t) applied to the modulation photogates, depending on the respective polarity or phase involved, to the accumulation gates which are biased with preferably double the dc voltage Ua and Ub. The modulation photogate voltages Uam(t) and Ubm(t) are preferably complementarily applied and are preferably composed of a bias voltage U0 and the modulation voltage +Um(t) and xe2x88x92Um(t) respectively superimposed in push-pull relationship. The two modulation photogates together preferably form a square surface. A pixel with only two modulation photogates can also be referred to as a dual pixel.
This principle according to the invention presupposes the photoelectric quantum effect, caused by electromagnetic waves. Nonetheless, reference will always be made to light waves, without this being interpreted as a limitation.
The actual mixing or multiplication process lies in the modulation voltage-dependent or phase-dependent drift of the photo-generated charge carriers to the right or to the left side of the modulation photogate (xe2x80x9ccharge swingxe2x80x9d). In that respect the charge difference between the charge carriers which are separated in that way and collected under the accumulation gates and transmitted to the electronic reading-out system, having regard to integration in a predetermined time, represents a measurement in respect of the correlation function of the envelope of the incident modulated light signal and the modulation voltage Um(t).
At the same time the charge sum of those charge carriers which have drifted to the accumulation gates and passed on remains uninfluenced by the position of the charge swing and is available as suitable pixel intensity or as pixel grey value.
In order to determine the relative phase or time delay of the incident light wave, it is necessaryxe2x80x94as described abovexe2x80x94to implement three measurements in respect of the three parameters dc voltage component and ac voltage component and relative phase. Therefore, it is possible to involve a configuration of the pixel of the photonic mixing element with three light-sensitive modulation photogates which are acted upon by modulation photogate voltages which involve three different phase shifts relative to the light wave irradiated by the transmitter.
Desirably however to determine the phase of the reception signal at each pixel of the photonic mixing element from the resulting correlation amplitudes, use is made of four different measurements in regard to four different phases of the mixer signal. That provides for over-determination, by means of which the noise can be significantly reduced.
The push-pull arrangement of the modulation photogate voltages at two modulation photogates per pixel provides that two respective ones of those measurements are implemented at the same time. Therefore, for example in the case of HF-modulation, it is sufficient to implement two measurements which are respectively displaced through 90xc2x0 at 0xc2x0/180xc2x0 and also at 90xc2x0/270xc2x0 phase difference in respect of the modulation photogate voltages Uam(t) and Ubm(t) respectively with respect to the phase of the radiated light in order to acquire the four different measurement values necessary.
A particularly preferred arrangement therefore is one in which the photonic mixing element respectively forming a pixel comprises four symmetrically arranged modulation photogates, wherein each two respective mutually oppositely disposed modulation photogates are acted upon with push-pull or 180xc2x0-phase-shifted modulation photogate voltages, wherein the two measurements which are respectively displaced through 90xc2x0 and which have been described hereinbefore in connection with the dual pixel, with a 0xc2x0/180xc2x0 and also 90xc2x0/270xc2x0 phase difference of the modulation photogate voltages, are implemented simultaneously in this case. Such a pixel can also be referred to as a quadruple pixel.
Furthermore, for calibration of the phase shift of the modulation photogate voltages Uam(t) and Ubm(t) it is preferably possible for a part of the light waves irradiated by the transmitter to be directed as a reference directly onto at least one of a plurality of pixels of an arrangement of a plurality of photonic mixing elements. The phase and amplitude information obtained from that directly irradiated pixel can then be used for the calibration operation or can be employed for adjustment of the phase shift to a predetermined value.
Conversely, in the case of independently excited, unknown modulation of the incident light wave radiated by an active object, by means of at least one photonic mixing element, it is possible to measure the light wave with the known high level of resolution of a lock-in amplifier. For that purpose the photonic mixing element together with a tunable modulation generator which is in place of the transmitter forms a phase-lock loop. In addition, both in lock-in amplification the phase-lock loop is used for example for HF-modulation and also the delay-lock loop is used for digital modulation.
For surveying passive objects, modulation of the irradiated light and corresponding modulation of the modulation photogate voltages Uam(t) and Ubm(t) respectively can be implemented in various ways. First of all it is possible to effect continuous HF-modulation, in which case the charge differences and the charge sums are read out repeatedly at intervals which can be retroactively influenced by the pixel intensity, for evaluation of the phase and amplitude information of the light wave.
An advantageous procedure is an intermittent mode of operation with pulse-form HF-modulation and lighting, for example in order for example in each case briefly to exceed an interference background lighting. In that case only the photo-generated charges are respectively integrated during the HF-pulse and then evaluated.
In determining in particular the phase or transit time information of reflected light waves, to increase the level of phase or transit time resolution, it is possible to use the HF-pulse compression process known from the radar art, with narrow correlation functions, for example the chirp procedure. In that case, both the modulation signal of the individual photonic mixing element and also the light wave of the transmitter, which lights with a predetermined phase relationship, and thus also the light wave reflected with the phase relationship being sought, is repetitively modulated with a chirp. By virtue of chirp modulation, in a suitable manner, the insertion of an adjustable delay between the modulation photogate voltage of the photonic mixing element and the light irradiated by the transmitter provides for resolving multiple targets or suppressing troublesome multiple reflections of a lit scene.
Pseudo-noise modulation (PN-modulation) described hereinafter is available as a further form of modulation, both as base band-PN- and also HF-PN-modulation. A sampling procedure with sample-and-hold operations in the case of repetitive light signals is a special case of mixing and correlation with needle pulses. The photonic mixing element according to the invention can advantageously be used in this case also and for other uses of pulsed modulation.
The modulation modes referred to are per se all known from the state of the art.
The charges which have drifted to the accumulation gates can now be the subject of further processing in various ways. On the one hand, the photonic mixing element can be constructed using CCD-technology, in which case the charges are collected or integrated beneath the accumulation gates and then displaced in conventional manner to the CCD-read-out circuit, for example in a three-phase shift cycle, and read out by way of p- or n-diffusion.
On the other hand, the photonic mixing element can be embodied using CMOS-technology as an active pixel element with pixel-specific electronic reading-out and signal pre-processing system. In that case, in practice the reading-out circuit which is conventional in CCD-engineering is taken at both respective sides directly to the modulation photogate. In that case the accumulation gates are preferably in the form of blocked low-capacitants pn-diodes and transmit the arriving photo-generated charges preferably directly by way of the electrodes Ga and Gb to the electronic pixel reading-out and signal pre-processing system for storage and processing there.
In the latter case therefore the two charge components of the charge swing are continuously read out and can be stored practically in a reaction-free manner, for example with a charge amplifier, on a respective downstream-connected capacitor.
The state of the art provides that, before each new measurement operation, the involved, charged-up capacitors are discharged by means of electronic reset switches and that desirably the fault voltages measured in the reset condition are used for correction of the actual measurement values. That use of the pixel-wise reaction-free reading-out procedure affords the advantage that the entire dynamics of the photonic mixing element and therewith the measuring method can be considerably enhanced in comparison with implementation using CCD-technology.
In a further preferred manner it is possible to directly compute the phase and amplitude information in an electronic pixel reading-out and signal pre-processing system, preferably in the form of on-chip integration. Such an application-specific opto-electronic chip (ASOC) or such an active pixel sensor (APS) enhances the measuring rate and permits pixel-wise pre-processing of the phases and/or amplitudes.
An important advantage of the present invention is that modulation is effected simultaneously with charge generation and separation. In other words, detection and mixing take place at the same time and without additional noisy and band-limiting intermediate stages. Therefore, the time drift errors which occur inter alia in the state of the art are prevented, the charge modulation and integration operations which are separated in terms of time and space from the detection operation necessarily occur and are not to be suppressed.
A further advantage of the present invention lies in the high limit frequency of the photonic mixing element. The limit frequency of charge transfer by the push-pull modulation voltage is comparable in terms of the maximum drift length or transfer distance, that is to say the sum length of the modulation photogates, with the limit frequency of corresponding MOS-transistors, and thus attains the GHz-range. In addition, troublesome common-mode signals are suppressed by virtue of anti-symmetrical charge carrier separation and difference formation. Each interference signal which does not correlate with the modulation signal, for example the background lighting, is suppressed in the charge difference, and that results in a high signal-to-noise ratio. Furthermore, there is only a slight time drift because of the combination of detection, mixing and charge carrier integration and difference formation on the same chip. In addition, a combination of practically all measurement functions becomes possible within a single semiconductor structure.
In comparison with the state of the art disclosed in DE 44 39 298 A1 with the use of Pockel cells as modulators, only low modulation voltages in the 1 instead of 1000 volt range are necessary. In addition, a 2D-arrangement of photonic mixing elements according to the invention ensures a large aperture on the receiver side.
In addition, no coherent or polarised light is required for determining the phase and/or amplitude information. Accordingly it is possible to use further specific properties of the incident light waves by the upstream arrangement of selective filters, for example in respect of polarisation and wavelength of the light. In addition, the arrangement affords a high level of sensitivity and a high signal-to-noise ratio by virtue of the elimination of the electronic mixers and wide-band photodetector amplifiers which are used in accordance with the state of the art.
The spectral optical band width of the light waves to be surveyed is determined by the spectral photosensitivity of the material used in the space charge zone under the photogates, that is to say for example in the case of silicon approximately the wavelength range of 0.3 to 1.1 xcexcm in the case of InGaAs about 0.8 to 1.6 xcexcm and in the case of InSb about 1 to 5.5 xcexcm.
The photonic mixing elements can be disposed in any zero-, one- or two-dimensional arrangement, and thus afford a wide spectrum of use geometries. In that respect, several 100,000 photonic mixing elements can be operated in parallel relationship with a modulation band width of for example 10-1000 MHz, so that for example a camera shot of a 3D-scene can be implemented extremely quickly, with determination of the distance information in each pixel. The phase image xcfx86(x,y) orxe2x80x94in the case of modulated lightingxe2x80x94the distance image or depth image with the radius vector or voxel distance R(x,y) is determined in pixel-wise manner by way of the charge differences of the charges which flow to the accumulation gates and which are read out. The corresponding charge sums afford the conventional pixel grey value A(x,y). The two can be combined to give the scaled grey value image or the 3D-image A(x,y,z).
In that respect, the 3D-image repetition rate is in the range of about 10 Hz to over 1000 Hz and depends on the number of photonic mixing elements used and the level of light intensity. By means of additional colour filters, it is possible to obtain the usual colour values red (x,y), green (x,y) and blue (x,y) of the distance image R(x,y).
The integrated structure of mixing and charge carrier integration not least also provides for a simple structure in respect of the photonic mixing element. Finally, there is no need to involve particular expense in the reception channel for a conventional optical imaging system is sufficient for imaging of the incident, possibly reflected light wave, if a one- or two-dimensional scene and not just a point is to be recorded. The measuring apparatus can be flexibly adapted to different 3D-scenes by virtue of synchronous zoom of the optical transmitting and receiving system.
In the case of the method according to the invention and the corresponding mixing element or an arrangement of a plurality of mixing elements, it is desirable if the pixel phase or the pixel transit time and the pixel brightness are ascertained directly by means of an active pixel sensor structure (APS) and then read out selectively or also serially preferably by way of a multiplex structure disposed on the same chip (the so-called on-chip multiplex structure). That increases the processing speed and also reduces the number of further components required.
If moreover pixel brightness is evaluated as the sum of the charges of the associated accumulation gates, as a grey value image, a particularly preferred embodiment of the invention is one which, in the case of background lighting, that is to say in the case of non-modulated lighting which is present beside the modulated lighting, eliminates by computation the charges produced by that additional lighting at the accumulation gates, by a procedure whereby the difference is formed between the grey value images which are achieved on the one hand with modulated lighting switched on and on the other hand without the modulated lighting, that is to say after, the modulated light source is switched off. No correlation information is contained in that basic brightness or that base amount of the charges at the accumulation gates so that the actual correlation information appears more clearly after subtraction of that base amount.
As already mentioned it is obviously appropriate if a plurality of the mixing elements are used either in a linear array, a surface array or a spatial array. In that respect, the term xe2x80x9clinearxe2x80x9d array is intended to mean not only a set of mixing elements which are arranged in a straight row in side-by-side or successive relationship but generally a set of mixing elements which are arranged along a line, wherein said line can be straight or also curved. In the case of the surface arrangements also, it is not only possible to provide planar mixing element arrangements in the form of a rectangular matrix even if that may also be preferable for practical reasons, but in principle the mixing elements can be arranged in accordance with any desired pattern and also on a curved surface, for example on the inside surface of a spherical shell. It is also possible to use arrays of the mixing elements on angled surfaces, that is to say simultaneously on two surfaces which include an angle with each other, and such arrangements are appropriate for given applications. Arrangements of that kind are embraced by the term xe2x80x9cspatial arrayxe2x80x9d.
In the case of such arrays comprising a plurality of and possibly several hundred or thousand mixing elements, an advantageous and desirable configuration of the method according to the invention is one in which at least one of the pixels or mixing elements is directly irradiated with a part of the intensity-modulated electromagnetic wave serving as lighting, in which case the measurement result obtained in that way is used at said at least one pixel for calibration of the other phases and brightness results. In that respect it is desirable if such a reference pixel is acted upon by the transmitter with selectively different levels of intensity or, for the situation where a plurality of reference pixels are used, each of those pixels is acted upon by a different level of intensity. That makes it possible to avoid errors which can possibly occur by virtue of the large dynamic scope of the measurement signals.
In the case of a one- or multi-dimensional mixing element arrangement of the above-indicated kind it is desirable if the pixels are constructed using MOS-technology on a silicon substrate and can be read out with a multiplex structure, preferably with a CCD-structure.
It will be appreciated that the mixing elements according to the invention are readily suitable for use in a digital photographic camera or video camera. For that purpose, it is only necessary to provide a suitable mixing element arrangement (for example in the form of a rectangular matrix) with integrated receiving optics, electronic evaluation system and signal processing for the difference signals, the sum signals and the associated reference signals, together with a digital memory for the grey value image computed therefrom, and the transit time or distance image. The arrangement also includes a suitable transmitter or a suitable light source which radiates a three-dimensional scene with modulated electromagnetic waves or modulated light, and transmitting optics which are suitably adjustable to the receiving optics, wherein all of those components are combined together to constitute a compact unit as a digital camera. In that respect the difference between a digital photographic camera and a digital video camera is essentially only that, in a corresponding video camera, a relatively large number of images has to be recorded and stored in correspondingly short intervals of time so that suitable devices must be provided for the storage and reproduction of corresponding image sequences.
It will be appreciated that in addition in all uses lighting or illuminating a scene can be implemented with modulated light from various spectral regions, so that the colour components or chromatic components of the images, which are obtained in that way, can be used to acquire and reconstruct complete colour images with the spatial depth information which is supplied at the same time.
For a higher band width and for example also for improved edge detection, it may be desirable to use a microlens optical system in which associated with each mixing element or pixel is a microlens optical system which reduces the incident light to the central region of the pixel so that deviations from the ideal potential configuration at the modulation gates which occur in particular in the edge regions of the photosensitive surfaces are practically removed. In addition, out-of-focus imaging, which is produced by means of the microlens optical system, in the detector plane of the mixing elements, can ensure that the imaging of edges, the imaging of which extends randomly in the centre between the two pixel halves, does not result in the generation of difference charges at the accumulation gates, which simulate a correlation or false depth information.
Arrays with the photomixing elements according to the invention are also highly suitable for detecting and possibly also tracking predetermined one-, two- or three-dimensional structures in the field of view of the arrangement in question and with consideration additionally being given to the depth information or the object distance of the object which is being sought and which is possibly to be tracked.
In specific terms, selectively determining the amplitudes and the displacement of the X-, Y- and the time coordinate T of the modulation signals by (xcex94X, xcex94Y, xcex94T) (wherein X and Y define two linearly independent coordinates which extend on the plane of a mixing element matrix and time T means the transit time delay of the modulation signals) provides for the implementation of a three-dimensional correlation, whereby a predetermined three-dimensional object is sought in space, detected and possibly tracked.
The photomixing element according to the invention additionally also has a wide area of application in the field of optical data transmission. In that respect the photomixing element according to the invention is simply used instead of a photodiode in a conventional optical signal receiver, possibly inclusive of signal regeneration, wherein the shape of the modulation signal is adapted in the optimum manner to the signal shape and the phase of the modulation signal is also adapted in optimum manner in a phase-lock loop to the phase position of the reception signal. In other words, the clock is obtained from the signal itself and used for optimum weighting of the reception signal, whereby the signal is separated in optimum fashion from the troublesome, noisy background. In that way, sensitivity and accuracy in respect of optical data transmission can be considerably improved in comparison with conventional photodiodes. This could in particular also permit a considerable increase in the length of the optical transmission sections without intermediate amplification and a higher number of parallel communication channels in a time, frequency and code multiplex mode.
Finally, the photomixing element according to the invention can also be used for example in optically-based position detection systems, wherein the mode of operation is in principle similar to that involved in the known GPS-system which permits very accurate determination of position by means of satellite transmitters which permit the encoded radiation of signals. In a corresponding optical position detection system the satellite transmitter which is known from the GPS-system would be replaced by a widely dispersive, modulated light source which is arranged correspondingly closer to the object whose position is to be determined, for example by means of laser diodes and an optical dispersion or scatter system, while the receiver is formed by one or more photomixing elements on the object, preferably by a plurality of photomixing elements which are oriented in various directions in order to detect the signals from light sources stationarily arranged at various points, with different modulations. In that case the encoded modulation permits a clear association of stationary light sources and the object whose position is to be determined, as well as the associated signal transit times, by means of which the position is determined.
A further use is that of a demultiplexer for optical data transmission. Encoding in the form of special modulation and the associated correlation by means of the photomixing element permits a clear association of various channels.
A further application and use of the high level of phase sensitivity of the photomixing elements according to the invention lies in the measurement of the Sagnac effect, that is to say the transit time or phase shift of light waves in rotating reference systems. For that purpose, modulated light is coupled into an optical fibre which is preferably laid in a plurality of turns and the output of the optical fibre lights one of the photomixing elements according to the invention. The modulation gates of that mixing element are modulated with the same frequency as the coupled-in light waves so that the correlation result in the form of the charge distribution at the photomixing element supplies a measurement in respect of the current frequency or phase shift. During each revolution of the reference system in which the axis of rotation is not in the plane of the turns of the optical fibre or optical waveguide, frequency and transit time and therewith also phase position change and are automatically detected by the photomixing element. It is worth noting in that respect that, with the photomixing element, such fibre gyro compass systems based on the Sagnac effect can now be embodied by means of incoherent light, which do not give rise to any problems in regard to their long-term stability as the corresponding sources of error in accordance with the state of the art, the high-frequency amplifier downstream of the optical detector and the electronic mixer, are completely eliminated.
In addition, besides absolute directional measurement which is made possible with such a system, it is also possible to effect speed measurement of a moving object by means of the photomixing element according to the invention, for example insofar as a part of the light waves is removed in a beam splitter before being introduced into the optical waveguide and is directed onto a stationary object, in which case the light reflected by the stationary object is captured by a suitable photomixing element receiver and evaluated in the manner which has already been described on a number of occasions, here in regard to the Doppler frequency shift.
Depending on the respective meaning and significance of the additional depth information of a line or matrix image, a given number of photomixing elements can be integrated in the appropriate technology in a CCD-, CMOS- or TFA (Thin Film on ASIC)-image sensor.
Furthermore, in the use of a 3D-line or matrix camera in accordance with the invention, it may be appropriate additionally to use a conventional 2D-camera, wherein a preferably spectral allocation and feed of the active modulated illumination component to the 3D-camera, and of the other unmodulated illumination component, is preferably effected with a beam splitter.
For uses of the photomixing elements for 3D-measurement or surveying, for greater distances for which the modulated lighting is too weak, it is possible to use a combination of at least two 3D-line- or matrix cameras, in which case in accordance with the invention measurement or surveying is effected in the near region on the basis of the transit time principle and in the far region on the basis of the triangulation principle with inter alia existing background lighting.
In that case depth measurement in the near region is implemented as described hereinbefore, in this case in parallel by way of at least two cameras.
For depth measurement in the far region; the optical axes of the cameras, which are formed by way of the PMD-chip centre point, are directed onto a common point of intersection in the volume region to be measured, for example by suitable PMD-chip displacement in the horizontal and vertical directions and in respect of the PMD-chip spacings, wherein at the same time focussing of the optical systems of the cameras are set to that distance. With suitable previous adjustment the pixel brightness values then coincide in that volume region of greatest depth sharpness.
For the detection and identification of the objects in that volume region, in the event of correspondence of the pixel amplitudes, the sum image of the photomixing elements is added by a briefly applied dc modulation voltage, in respect of the difference image, associated with the set distance data and evaluated, while the non-corresponding pixel amplitudes are removed in the difference image by a modulation voltage which is set to zero, Uma=Umb=0.
In that way, by means of angle scanning, the 3D-scene is also measured and surveyed outside the range of the modulated transmitter lighting, wherein the necessary angles are attained both by suitable displacement of the PMD-chips and also by rotation of the individual stereo cameras and/or by pivotal movement of the entire arrangement.
The many possible uses of which only some are herein described in part in detail and in part only briefly indicated are also to be found in the following list setting forth further possible uses, the further description of which would go beyond the scope of the present application, in which respect the following list is also in no way exhaustive.
More specifically possible uses are meaningful and can be envisaged in the following areas:
Digital 3D-photographic camera,
Digital 3D-video camera,
Danger area monitoring,
Security engineering and xe2x80x9cintelligent buildingsxe2x80x9d,
Occupant detection and identification in motor vehicles, xe2x80x9cintelligent air bagxe2x80x9d,
Electronic 3D-rearview mirror,
Recognition of the traffic situation in road traffic,
Autonomous vehicle navigation,
Incoherent fibre gyro and Doppler speed measurement,
Control of autonomous transport vehicles,
Industrial cleaning robots,
Personal identification, authentification and checking of access authority,
Identification of objects, for example vehicles,
Production monitoring, material testing, 100% quality testing,
Electronic xe2x80x9c3D-eyexe2x80x9d for a robot hand robust, small, all solid state,
Vehicle speed and distance-covered measurement, road condition detection, traffic jam,
Track free signalling, contact wire monitoring on railways,
Medical engineering, endoscopy,
CDMA-engineering for optical free-space or line communication,
Interactive 3D-communication for example in the multimedia area, and
3D-measurement of moving objects with a line of photomixing elements.
In that respect the following advantages of the photomixing elements of the present invention are to be emphasised (abbreviated hereinafter as xe2x80x9cPMDxe2x80x9d standing for xe2x80x9cPhotonic Mixer Devicexe2x80x9d):
1. PMD combines: detection, push-pull mixing and integration in a very small space of 1/100-1/1000 mm2xe2x86x92electro-optical correlation.
2. 2-times/4-times-PMD: substitute for 2 or 4 expensive wide-band amplifiers with high dynamics and group transit time constancy and for 2 and 4 electronic mixers respectively.
3. The high level of electronic cross-talk sensitivity between transmitter and receiver is eliminated.
4. High level of integratability with some 100,000 parallel electro-optical modulators.
5. A PMD-3D-photographic or video camera is fully integratable, small, light, robust and flexibly adaptable by an optical zoom system for light transmitter and receiver. Measurement volumes for natural surfaces, distances of about 20 cm to 50 m with aperture angles of about 5xc2x0 to 50xc2x0.
6. Extremely fast 3D-image recording in the 10 Hz-1000 Hz-range. Sensitivity and S/N-ratio correspond to presentday CCD- and CMOS-cameras.
7. The expected depth resolution is about 0.5 mm to 50 mm depending on the respective measurement time, lighting intensity, optics involved and spacing by virtue of optimum reference.
8. Maximum band width according to respective pixel size up to the GHz-range.
9. Modulation voltages in the range of less than 1 volt.
10. No coherent, polarised or narrow-band light is required and the spectral range depends on the light-sensitive material (for example in the case of InSb up to 5.5 xcexcm).
11. Simultaneous recording of the 3D-depth image and the 2D-grey value image, by virtue of data fusion, affords optimised evaluation of the 3D-grey value image (or 3D-colour image).
12. The read-out circuit, by virtue of intensity-dependent variation in the integration time Ti, permits an increase in the dynamics by about 8 bits (factor 256).